Sensors positions determinations

ABSTRACT

In an example, an apparatus includes a first sensor, a second sensor, and a controller. The first sensor collects a first set of data describing a three-dimensional object that is placed in a constant position in a scene. The second sensor collects a second set of data describing the three-dimensional object. The first sensor has a first position in the scene, while the second sensor has a second position in the scene. The controller generates a first rendering of the three-dimensional object from the first set of data and generates a second rendering of the three-dimensional object from the second set of data. The controller also determines the first position and the second position based on an alignment of the first rendering and the second rendering with a three-dimensional model of the three-dimensional object.

BACKGROUND

A three-dimensional (3D) scanning system collects data on the physicalcharacteristics (e.g., shape, size, weight, and/or color) of a 3D objector environment. The collected data can then be used to construct adigital 3D model of the object or environment. For instance, a 3Dscanning system may be used by a law enforcement agency to document acrime scene, by a video game developer to render a digital object orcharacter, by a real estate agent to construct a “virtual tour” of aproperty for sale, by an archaeologist to replicate a cultural artifact,and in other applications.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a high-level block diagram of an examplethree-dimensional scanning system to determine positions of sensors ofthe three-dimensional scanning system via a three-dimensional object;

FIG. 2 illustrates a flow diagram of an example method of operation at athree-dimensional scanning system to determine positions of sensors ofthe three-dimensional scanning system; and

FIG. 3 depicts a high-level block diagram of an example electronicdevice to determine positions of sensors of a three-dimensional scanningsystem via a three-dimensional object.

DETAILED DESCRIPTION

The present disclosure broadly describes an apparatus, method, andnon-transitory computer-readable medium for determining the positions ofsensors in a three-dimensional (3D) scanning system. As discussed above,a 3D scanning system uses a plurality of sensors to collect data on thephysical characteristics (e.g., shape, size, weight, pressuredistribution, and/or color) of a 3D object or environment. The collecteddata can then be used to construct a digital 3D model of the object orenvironment. The accuracy of the 3D model depends in part on knowledgeof the relative positions (i.e., locations and/or orientations) of thesensors, which allows the data collected by the various sensors to befused into a single model that is a true 3D representation of the 3Dobject or location.

Examples of the present disclosure determine the respective and/orrelative positions of a plurality of sensors in a 3D scanning system,using a single 3D calibration object having known geometry, dimensions,and/or color. In one example, a 3D model is generated of the 3Dcalibration object prior to scanning. The 3D calibration object is thenpositioned in the 3D scanning system. Each of the sensors of the 3Dscanning system scans the 3D calibration object, and data collectedthrough the scanning allows a controller to generate renderings of thecalibration object from the respective points of view of the individualsensors (where any given rendering may or may not include the entire 3Dcalibration model). The 3D model is then aligned to each rendering, andthe respective positions (e.g., locations and/or orientations) of thesensors are derived through sets of transformation parameters. Once theposition of each sensor is known, the plurality of sensors can be usedtogether to generate accurate 3D models of 3D objects.

Within the context of the present disclosure, the “position” of a sensoris understood to indicate the location and/or orientation of the sensorin a 3D scanning system. The “location” of a sensor may refer to thesensor's linear position in a 3D space. The “orientation” of a sensormay refer to the sensor's angular position in the 3D space.

FIG. 1 depicts a high-level block diagram of an examplethree-dimensional scanning system 100 to determine positions of sensorsof the three-dimensional scanning system via a three-dimensional object.As shown in FIG. 1, the electronic device or system 100 generallycomprises a controller 102 and a plurality of sensors including a firstsensor 104 ₁ and a second sensor 104 ₂ (hereinafter collectivelyreferred to as “sensors 104”).

Although two sensors 104 are illustrated, the system 100 may include anynumber of sensors. As discussed above, the sensors 104 collect data onthe physical characteristics (e.g., shape, size, weight, pressuredistribution, and/or color) of a 3D target (e.g., an object or anenvironment). The sensors 104 may include different types of sensors,such as 3D cameras, color cameras, thermal cameras, depth sensors,pressure plates, and/or other types of sensors. For instance, if thesystem 100 is used to design custom orthotics, then the sensors mayinclude pressure plates to measure foot pressure (e.g., to quantify theflatness, weight distribution, or other characteristics of the foot) aswell as 3D cameras to measure foot size and geometry. The respectivefields of view of the sensors 104 may be overlapping or non-overlapping.

The controller 102 is communicatively coupled to the sensors 104 anduses the data collected by the sensors 104 to generate a 3D model of the3D target. In some examples, the controller 102 may also send signals tothe sensors 104 to control the positions of the sensors 104, in order toimprove the data collection process. Before the 3D target is scanned,the controller 102 in one example determines the position (e.g.,location and/or orientation) of each of the sensors 104. Knowing therelative positions of the sensors 104 helps the controller 102 to fusethe data collected from the sensors 104 into a single, accurate 3D modelof the 3D target.

In one example, the positions of the sensors 104 are determined througha calibration process using a 3D calibration object 110 having knowngeometry and dimensions. The known geometry and dimensions of the 3Dcalibration object 110 may be described in a 3D model 108 that is madeavailable to the controller 102. In addition, the 3D calibration object110 may have certain physical characteristics that are designed to aidin determining the positions of specific types of sensors. For instance,an infrared reflective calibration object may be used to determine theposition of an infrared camera, a weighted calibration object may beused to determine the position of a pressure plate, a texturedcalibration object may be used to determine the position of a colorsensor, and so on.

The 3D calibration object 110 may be placed in a constant or fixedposition in the scene, e.g., such that the location and orientation ofthe 3D calibration object 110 does not change until the calibrationprocess is finished. Once the 3D calibration object 110 is placed in itsconstant position, the first sensor 104 ₁ scans the 3D calibrationobject 110 from a first position in the scene, while the second sensor104 ₂ scans the 3D calibration object from a second position in thescene. The first sensor 104 ₁ and the second sensor 104 ₂ send the dataobtained by the scanning to the controller 102. Based on this data, thecontroller 102 produces a first rendering 106 ₁ that depicts thephysical characteristics of the 3D calibration object 110 from theperspective of the first sensor 104 ₁ and second rendering 106 ₂ thatdepicts the physical characteristics of the 3D calibration object 110from the perspective of the second sensor 104 ₂. The first sensor 104 ₁and the second sensor 104 ₂ may operate simultaneously to scan the 3Dcalibration object 110, or the first sensor 104 ₁ and second sensor 104₂ may operate one at a time (e.g., the second sensor 104 ₂ does notbegin scanning until the first sensor 104 ₁ is done scanning).

Both the first rendering 106 ₁ and the second rendering 106 ₂ are usedby the controller 102, along with the 3D model 108 of the 3D calibrationobject 110, to determine the positions of the sensors 104. One exampleof a method for determining the positions of the sensors 104 using thisinformation is described in greater detail with respect to FIG. 2.

FIG. 2 illustrates a flow diagram of an example method 200 of operationat a three-dimensional scanning system to determine positions of sensorsof the three-dimensional scanning system. The method 200 may beperformed, for example, by the controller 102 of FIG. 1. As such,reference may be made in the discussion of the method 200 to componentsof the 3D scanning system 100 of FIG. 1. However, such references aremade for the sake of example, and are not intended to be limiting.

The method 200 begins in block 202. In block 204, a 3D model of a 3Dobject, e.g., a 3D calibration object, is obtained. The 3D modeldescribes the geometry and dimensions of the 3D calibration object, aswell as potentially other physical characteristics of the 3D calibrationobject (e.g., color, weight, etc.). The 3D model may be obtained throughprecision manufacturing (e.g., from computer-aided design data for the3D calibration object), through 3D scanning of the 3D calibration objectby a calibrated 3D scanning system, or through other reliable means.

In block 206, a first rendering of the 3D calibration object isgenerated from a first set of data describing the 3D calibration object(e.g., describing a physical characteristic of the 3D calibrationobject). The first set of data may be collected by a first sensor. Thefirst sensor has a first position in a 3D scanning system in which the3D calibration object is placed in a constant position (e.g., someposition having a convenient origin point within the 3D scanningsystem). From this first position, the first sensor has a first field ofview that allows it to detect and scan the physical characteristics ofat least part of the 3D calibration object. The type of physicalcharacteristics detected by the first sensor depends on the type of thefirst sensor. For instance, the first sensor may comprise a 3D camera, acolor camera, a thermal camera, a depth sensor, a pressure plates,and/or another types of sensor.

In block 208, a second rendering of the 3D calibration object isgenerated from a second set of data describing the 3D calibration object(e.g., describing a physical characteristic of the 3D calibrationobject). The second set of data may be collected by a second sensor. Thesecond sensor has a second position in the 3D scanning system in whichthe 3D calibration object is placed. From this second position, thesecond sensor has a second field of view that allows it to detect andscan the physical characteristics of at least part of the 3D calibrationobject. The second field of view may or may not overlap with the firstfield of view. The type of physical characteristics detected by thesecond sensor depends on the type of the second sensor. For instance,like the first sensor, the second sensor may comprise a 3D camera, acolor camera, a thermal camera, a depth sensor, a pressure plates,and/or another types of sensor. In one example, the first sensor and thesecond sensor may comprise different types of sensors. For instance, thefirst sensor may comprise a 3D camera, while the second sensor comprisesa pressure sensor. In one example, the first set of data and the secondset of data are collected simultaneously by the first sensor and thesecond sensor; however, in another example, the first set of data andthe second set of data are obtained at different times. However, theposition of the 3D calibration object remains constant and does notchange between data collection/scanning by the first sensor and thesecond sensor.

In block 210, the first rendering and the second rendering are alignedto the 3D model of the 3D calibration object. For instance, the firstrendering may be aligned to a first portion of the 3D model that it mostclosely matches, while the second rendering may be aligned to a secondportion of the 3D model that it most closely matches. In one example,the first rendering and the second rendering may overlap. That is,certain portions of the 3D calibration object may be depicted in boththe first rendering and the second rendering. In one example, thealignment is a two-stage process that involves a first rough, globalalignment (e.g., using a 4-points congruent sets algorithm, a super4-points congruent sets algorithm, a random sample consensus algorithm,or a bundle adjustment algorithm) followed by a second, finer alignment(e.g., using an iterative closest point algorithm or a bundle adjustmentalgorithm).

In block 212, the position of the first sensor is identified based onthe alignment of the first rendering to the 3D model of the 3Dcalibration object. For instance, the alignment of the first renderingto the 3D model may allow a first set of transformation parameters to bederived, where the first set of transformation parameters describes aposition of the first sensor relative to the 3D model. The first set oftransformation parameters may comprise, for example, a homogeneoustransformation matrix (e.g., a 4×4 matrix used to simultaneouslyrepresent rotation and translation), a set of Cartesian coordinates, orthe like.

In block 214, the position of the second sensor is identified based onthe alignment of the second rendering to the 3D model of the 3Dcalibration object. For instance, the alignment of the second renderingto the 3D model may allow a second set of transformation parameters tobe derived, where the second set of transformation parameters describesa position of the second sensor relative to the 3D model. The second setof transformation parameters may comprise, for example, a homogeneoustransformation matrix, a set of Cartesian coordinates, or the like.

In block 216, the positions of the first sensor and the second sensorare stored. In one example, storage of a sensor's position involvesstoring coordinates that describe the sensor's position (e.g., locationand/or orientation). Storage of the sensor's position may also involvestoring the set of transformation parameters associated with thesensor's position, so that the set of transformation parameters can beused at a later time, e.g., to correct data collected by the sensor withrespect to a new 3D target object to be modeled.

The method 200 ends in block 218.

Once the positions of the sensors have been determined, the 3D scanningsystem may be ready to scan new 3D target objects. Knowing therespective positions of the sensors within the 3D scanning system allowsthe 3D scanning system to properly fuse data collected by the sensorsinto accurate 3D models of the 3D target objects. For instance, thetransformation parameters for each of the sensors may be used to guidefusion of the data collected from the sensors (e.g., to correct theposition of data or the manner in which two or more pieces of data arefused).

Once the transformation parameters for the individual sensors arederived, transformations between sensors can also be derived. Forexample, knowing the first set of transformation parameters for thefirst sensor and the second set of transformation parameters for thesecond sensor may allow one to determine the position of the firstsensor relative to the second sensor.

It should be noted that although not explicitly specified, some of theblocks, functions, or operations of the method 200 described above mayinclude storing, displaying and/or outputting for a particularapplication. In other words, any data, records, fields, and/orintermediate results discussed in the method 200 can be stored,displayed, and/or outputted to another device depending on theparticular application. Furthermore, blocks, functions, or operations inFIG. 2 that recite a determining operation, or involve a decision, donot imply that both branches of the determining operation are practiced.In other words, one of the branches of the determining operation may notbe performed, depending on the results of the determining operation.

FIG. 3 depicts a high-level block diagram of an example electronicdevice 300 to determine positions of sensors of a three-dimensionalscanning system via a three-dimensional object. For instance, thecontroller 102 illustrated in FIG. 1 may be configured in a mannersimilar to the electronic device 300. As such, the electronic device 300may be implemented as a controller of an electronic device or system,such as a three-dimensional scanning system.

As depicted in FIG. 3, the electronic device 300 comprises a hardwareprocessor element 302, e.g., a central processing unit (CPU), amicroprocessor, or a multi-core processor, a memory 304, e.g., randomaccess memory (RAM) and/or read only memory (ROM), a module 305 fordetermining the positions of sensors in a 3D scanning system, andvarious input/output devices 306, e.g., storage devices, including butnot limited to, a tape drive, a floppy drive, a hard disk drive or acompact disk drive, a receiver, a transmitter, a display, an outputport, an input port, and a user input device, such as a keyboard, akeypad, a mouse, a microphone, and the like.

Although one processor element is shown, it should be noted that theelectronic device 300 may employ a plurality of processor elements.Furthermore, although one electronic device 300 is shown in the figure,if the method(s) as discussed above is implemented in a distributed orparallel manner for a particular illustrative example, i.e., the blocksof the above method(s) or the entire method(s) are implemented acrossmultiple or parallel electronic devices, then the electronic device 300of this figure is intended to represent each of those multipleelectronic devices.

It should be noted that the present disclosure can be implemented bymachine readable instructions and/or in a combination of machinereadable instructions and hardware, e.g., using application specificintegrated circuits (ASIC), a programmable logic array (PLA), includinga field-programmable gate array (FPGA), or a state machine deployed on ahardware device, a general purpose computer or any other hardwareequivalents, e.g., computer readable instructions pertaining to themethod(s) discussed above can be used to configure a hardware processorto perform the blocks, functions and/or operations of the abovedisclosed method(s).

In one example, instructions and data for the present module or process305 for determining the positions of sensors in a 3D scanning system,e.g., machine readable instructions can be loaded into memory 304 andexecuted by hardware processor element 302 to implement the blocks,functions or operations as discussed above in connection with the method200. For instance, the module 305 may include a plurality of programmingcode components, including an alignment component 308 and atransformation component 310. These programming code components may beincluded, for example, in a controller of a 3D scanning system, such asthe controller 102 of FIG. 1.

The alignment component 308 may be configured to align a rendering of a3D calibration object produced based on data collected by a sensor witha 3D model of the 3D calibration object. For instance, the alignmentcomponent 308 may be configured to perform all or part of block 210 ofthe method 200.

The transformation component 310 may be configured to identify alocation of a sensor in a 3D scanning system, e.g., by deriving a set oftransformation parameters that describe the sensor's location relativeto a 3D calibration object. For instance, the transformation component310 may be configured to perform all or part of blocks 212-214 of themethod 200.

Furthermore, when a hardware processor executes instructions to perform“operations”, this could include the hardware processor performing theoperations directly and/or facilitating, directing, or cooperating withanother hardware device or component, e.g., a co-processor and the like,to perform the operations.

The processor executing the machine readable instructions relating tothe above described method(s) can be perceived as a programmed processoror a specialized processor. As such, the present module 305 fordetermining the positions of sensors in a 3D scanning system, of thepresent disclosure can be stored on a tangible or physical (broadlynon-transitory) computer-readable storage device or medium, e.g.,volatile memory, non-volatile memory, ROM memory, RAM memory, magneticor optical drive, device or diskette and the like. More specifically,the computer-readable storage device may comprise any physical devicesthat provide the ability to store information such as data and/orinstructions to be accessed by a processor or an electronic device suchas a computer or a controller of a 3D scanning system.

It will be appreciated that variants of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be combined intomany other different systems or applications. Various presentlyunforeseen or unanticipated alternatives, modifications, or variationstherein may be subsequently made which are also intended to beencompassed by the following claims.

What is claimed is:
 1. An apparatus, comprising: a first sensor tocollect a first set of data describing a three-dimensional object thatis placed in a constant position in a scene, wherein the first sensorhas a first position in the scene; a second sensor to collect a secondset of data describing the three-dimensional object, wherein the secondsensor has a second position in the scene; and a controller to generatea first rendering of the three dimensional object from the first set ofdata, to generate a second rendering of the three dimensional objectfrom the second set of data, and to determine the first position and thesecond position based on an alignment of the first rendering and thesecond rendering with a three-dimensional model of the three-dimensionalobject.
 2. The apparatus of claim 1, wherein the first sensor is adifferent type of sensor than the second sensor.
 3. The apparatus ofclaim 1, wherein a field of view of the first sensor and a field of viewof the second sensor are non-overlapping.
 4. The apparatus of claim 1,wherein the first position indicates a location of the first sensor andan orientation of the first sensor, and the second position indicates alocation of the second sensor and an orientation of the second sensor.5. A non-transitory machine-readable storage medium encoded withinstructions executable by a controller of an electronic device, themachine-readable storage medium comprising: instructions to align athree-dimensional model of a three-dimensional object to a firstrendering of the three-dimensional object that is produced based on afirst set of data collected by a first sensor of the electronic devicehaving a first position in a scene; instructions to align thethree-dimensional model to a second rendering of the three-dimensionalobject that is produced by based on a second set of data collected by asecond sensor of the electronic device having a second position in thescene, wherein a position of the three-dimensional object in the sceneremains constant during collection of the first set of data and thesecond set of data; instructions to identify the first position, basedon aligning of the three-dimensional model to the first rendering; andinstructions to identify the second position, based on aligning of thethree-dimensional model to the second rendering.
 6. The non-transitorymachine-readable storage medium of claim 5, wherein the first sensor isa different type of sensor than the second sensor.
 7. The non-transitorymachine-readable storage medium of claim 5, wherein a field of view ofthe first sensor and a field of view of the second sensor arenon-overlapping.
 8. The non-transitory machine-readable storage mediumof claim 5, wherein the first position indicates a location of the firstsensor and an orientation of the first sensor, and the second positionindicates a location of the second sensor and an orientation of thesecond sensor.
 9. The non-transitory machine-readable storage medium ofclaim 5, wherein the first set of data and the second set of data arecollected simultaneously by the first sensor and the second sensor. 10.The non-transitory machine-readable storage medium of claim 5, whereinthe instructions to align the three-dimensional model of athree-dimensional object to the first rendering comprises: instructionsto perform a global alignment of the three-dimensional model of athree-dimensional object to the first rendering; and instructions toperform, subsequent to performing the global alignment, a fineralignment of the three-dimensional model to the first rendering.
 11. Thenon-transitory machine-readable storage medium of claim 5, wherein theinstructions to identify the first position comprise instructions toderive a first set of transformation parameters that describes aposition of the first sensor relative to the three-dimensional object.12. The non-transitory machine-readable storage medium of claim 11,wherein the first set of transformation parameters comprises ahomogeneous transformation matrix.
 13. The non-transitorymachine-readable storage medium of claim 11, wherein the first set oftransformation parameters comprises a set of Cartesian coordinates. 14.A method, comprising: aligning, by a controller of an electronic device,a three-dimensional model of a three-dimensional object to a firstrendering of the three-dimensional object that is based on a first setof data collected by a first sensor of the electronic device having afirst position in a scene; aligning, by the controller, thethree-dimensional model of the three-dimensional object to a secondrendering of the three-dimensional object that is based on a second setof data collected by a second sensor of the electronic device having asecond position in the scene, wherein a position of thethree-dimensional object in the scene remains constant during collectionof the first set of data and the second set of data; identifying, by thecontroller, the first position, based on aligning of thethree-dimensional model to the first rendering; and identifying, by thecontroller, the second position, based on aligning of thethree-dimensional model to the second rendering.
 15. The method of claim14, wherein a field of view of the first sensor and a field of view ofthe second sensor are non-overlapping.